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I . INTRODUCTION 


In  this  paper  the  eigenvalues  and  the  eigenvectors  of  a singular 

matrix.  A,  are  investigated.^  Present  techniques  for  determining  the 
eigenvalues  of  a matrix  require  that  the  matrix  be  nonsingular  or  that 
some  iterative  procedure  such  as  root  searching  techniques  be  applied  to 
the  characteristic  equation.  Here,  a computational  (non-iterative  type) 
algorithm  is  given  to  construct  a (smaller)  nonsingular  matrix,  Q,  that 
has  exactly  the  same  nonzero  eigenvalues  as  A and  only  these  eigenvalues. 

2 

Thus,  a standard  technique  such  as  the  QR  algorithm  can  be  applied  to 
the  matrix  Q to  evaluate  the  eigenvalues  and  eigenvectors  of  Q. 

The  algorithm  given  in  this  paper  yields  the  eigenvectors  of  the 
nonzero  eigenvalues  of  A,  which  are  obtained  from  the  eigenvectors  of 
Q,  and  the  rank  of  the  matrix  A.  The  eigenvectors  associated  with  the 
zero  eigenvalues  of  A are  an  immediate  consequence  of  the  algorithm. 


II.  MOTIVATION  OF  THE  ALGORITHM 


Let  A = (a. .)  be  an  nxn  matrix  and  U an  nxn  nonsingular  matrix. 
-1 

Define  W = U AU.  Let  X and  X be  an  eigenvalue  and  an  eigenvector, 
respectively,  of  A,  then 


AX  = XX 

U‘^A(UU'^)X  = XU'^X  (1) 

W(U"^X)  = X(U"^X), 


that  is,  X and  U”^X  are  an  eigenvalue  and  eigenvector  of  W. 
if  p and  Z are  an  eigenvalue  and  an  eigenvector  of  W,  then  p 
an  eigenvalue  and  an  eigenvector  of  A. 


Conversely, 
and  UZ  are 


Suppose  A is  of  rank  r and  there  exists  a U such  that  W can  be 
partitioned  into  the  following  form: 


^The  algorithm  given  in  this  paper  was  developed  by  the  author  in 
response  to  a problem  posed  by  H.  McCoy,  TRASANA,  to  obtain  the 
eigenvalues  and  eigenvectors  of  a special  singular  matrix. 


2 

Wilkinson,  J.  H. , The  Algebfaic  Eigenvalue  Problem , Clarendon  Press, 
Oxford,  1965. 
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(2) 


W = U"^AU 


> 


where  is  an  rxr  matrix,  0^  and  0^  are  rx(n-r)  and  (n-r)x(n-r)  zero 

matrices,  respectively,  and  B is  an  (n-r) xr  matrix.  Let  X ^ 0 be  an 
eigenvalue  o£  W,  and  of  A,  and  Z be  the  associated  eigenvector.  If  the 
vector  Z,  as  a matrix,  is  partitioned  into  Z^,  a rxl  matrix,  and  Y,  a 

(n-r) xl  matrix,  then  we  can  write  the  following  matrix  equation 


as 


and 


Q Z = XZ 
T r r 


BZ  = XY, 
r 


(3) 

C4) 

(5) 


that  is,  X and  Z^  are  an  eigenvalue  and  eigenvector  of  the  matrix  Q^. 

Thus,  the  nonzero  eigenvalues  o£  W,  therefore  for  A,  are  the  eigenvalues 
(nonzero)  of  Q^.  The  associated  eigenvectors  of  A are  given  by 


(6) 


which  are  determined  from  the  eigenvectors  of 


Conversely,  if  X 0 is  an  eigenvalue  of  and  Z^  the  eigenvector, 

then  it  follows  that  X,  which  is  defined  by  (6) , is  an  eigenvector  of  A 
and  X is  the  eigenvalue. 


The  eigenvectors  of  A associated  with  the  zero  eigenvalues  are 
immediate.  Define  the  column  vectors 


E. 

1 


1 ,r+i> 
^2,r+i 


,6' 

n,r+i 


i = 1 , 2,  . . . , n-r. 


(7) 
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where  6 ^ is  the  Kronecker  delta  function.  Then  WE.  = 0,  that  is,  E., 


1 = li 


kl 

2, 


. , n-r,  are  the  associated  eigenvectors  for  the  zero 


eigenvalues  of  W.  Thus,  LIE.,  i = 1,  2, 


. , n-r,  are  the  eigenvectors 


of  the  zero  eigenvalues  of  A;  but  these  eigenvectors  are  just  the  column 
vectors  represented  by  the  last  n-r  columns  of  the  matrix  U. 


Since  the  rank  of  A is  r,  it  follows  that  {UE^}  is  the  total  set 

of  (distinct)  eigenvectors  associated  with  the  zero  eigenvalue.  If 

is  singular,  this  implies  that  the  characteristic  equation  has  a zero 
of  multiplicity  greater  than  n-r;  but  no  greater  collection  of  eigen- 
vectors. The  process  would  then  be  applied  to  to  determine  a matrix 

of  smaller  order  that  has  the  same  nonzero  eigenvalues  as  A. 


III.  EXISTENCE  OF  THE  MATRIX  U 


We  will  now  show  the  existence  of  the  matrix  U.  Let  A. , 

1 

i = 1,  2,  . . .,  n,  denote  the  column  vectors  represented  by  the  columns 
of  the  matrix  A.  The  definition  and  notation  for  the  inner  product  of 
any  two  vectors  is  given  by 


(8) 


where  a denotes  the  complex  conjugate  of  a. 

We  will  now  apply  the  Gram-Schmidt  orthogonalization  process 
(see  [3])  to  the  set  of  vectors  (A^}.  If  A^  is  the  zero  vector,  then 

interchange  A^  and  A^.  Let  be  the  elementary  matrix  that  interchanges 

column  one  and  column  n,  when  post  matrix  multiplication  is  applied 
(this  is  the  identity  matrix  with  the  first  and  nth  columns  interchanged) . 
If,  after  this  interchange,  A^  is  zero,  then  interchange  A^^  and  A 

where  is  the  appropriate  elementary  matrix.  Continue  until  A^  is 

not  the  zero  vector.  Define  = A^  and 


3 

Berberian,  S.  K.,  Introduction  to  Hilbert  Space  . Oxford  University 
Press,  New  York,  1961. 
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(9) 


^^2  ^ '^2  " “21^1  ’ 


where  a 


21 


(A„,V  ) 


If  V2  is  the  zero  vector,  interchange  and  where  was 

the  last  vector  interchanged;  and  the  appropriate  elementary  matrix. 

Define  'to  be  the  elementary  matrix  that  adds  ““21  'titles  the  second 

column  to  column  k.  This  is  given  by  appending  to  the  identity  matrix 
-a2j^  in  row  2 and  column  k.  Construct  a new  ^2  zero 

vector,  then  continue  in  the  construction  until  a zero  vector  is 
generated,  that  is,  define 


i-1 


V.  = A 
1 


. - yi ct.  .V. , 

^ j=i  ^ 


(10) 


(A^,V.) 
“ij  = Tv^ 


j = 


i-1, 


If  V.  is  the  zero  vector, 
1 


1 y 

interchange  with  the  last  vector  not  interchanged,  say  A^,  and  assign 


an  appropriate  elementary  matrix  U,  . Since  V.,  j = 1,  . . .,  i-1,  can 

K ') 

be  written  in  terms  of  A^^, 
following  form: 


A.,  then  A.  can  be  written  in  the 
J 1 


A. 

1 


+ g.  . -A.  - . 
1,1-1  1-1 


(11) 


Furthermore,  let  U,  . , j = 1,  . 

K+J 


.,  i-1,  be  the  elementary  matrices 
column  m.  Contini 
was  the  last  vector  interchanged. 


which  adds  -g. . times  column  i to  column  m.  Continue  this  construction 
ij 

for  V.  until  A.  , 

1 1+1 


Define  U to  be  the  product  of  the  elementary  matrices  generated 
above,  that  is. 


u = 


u. 


(12) 


Then  AU  has  the  form: 


AU  = 


r "■) 

\R  0-/ 


(13) 
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For  example,  if  is  the  zero  vector,  then 


A.  = 
1 


i-1 


j = l 


3.  .A. 
11  1 


(14) 


The  product  of  the  associated  elementary  matrices,  say 

would  interchange  column  i with  column  m and  substract  3^j  times  column 

j,  j = 1,  . . .,  i-1,  from  column  m.  This  would  result  in  a zero  column 
in  column  m. 


If  Up  interchanges  column  i with  column  j,  then  premultipliction 


by  U * would  interchange  row  i-with  row  j;  and  if  U would  add  -3 

P *1  _ _ 

times  column  i to  column  j,  would  add  3 times  row  j to  row  i. 


Thus , 


U“^  = U“^  . . . U2^U“^  (15) 

would  result  in  row  operations.  Therfore,  U ^AU  would  have  the  same 
form  as  (13),  that  is. 


U“^AU 


(16) 


The  resulting  collection  of  nonzero  vectors,  A^^,  . . .,  A^,  under 

the  Gram-Schmidt  process,  is  the  largest  collection  of  independent 
vectors  represented  by  the  columns  of  A;  therefore,  r is  the  rank  of  A. 


IV.  CONSTRUCTION  OF  THE  MATRIX  U 


In  this  section  we  will  develop  a computationally  feasible  algorithm 
for  the  construction  of  the  matrices  U,  Q and  B.  It  should  be  noted 

-1 

that  the  actual  construction  of  U will  not  be  required  to  obtain  the 
final  result. 

To  start  the  construction,  set  U = (u^j)  = (6^^),  the  identity 
matrix.  If  there  exists  a vector  equal  to  the  zero  vector,  then 
from  (14) 


9 


(18) 


i-1 

L = y)  e. .A.  . 


Assume  A , was  the  last  column  interchanged.  If  u. . 

m+1  11 


1,  set  u. . =0, 
11 


0,  u = 1, 

m+l,m 


u.  = 1,  u . = 1 and  u =0.  If  u. . = 0,  set  u . 
im  mi  mm  ii  m+l,i 

u . = 1 and  u =0.  This  interchanges  column  i and  column  m. 
mi  mm 

Simultaneously,  replace  column  i with  column  m in  matrix  A.  Note  that 
column  m need  not  be  replaced  by  column  i,  for  this  column  is  assumed 
to  be  the  zero  vector.  In  order  to  accomplish  this,  add  , 

j = 1,  . . .,  i-1,  to  column  m in  the  U matrix,  where  the  ith  row  is 
determined  from  the  one  and  only  one  nonzero  element  in  the  jth  column 
of  the  U matrix.  This  nonzero  element  is  unity.  This  construction  is 
continued  for  i until  i+1  was  the  last  column  interchanged.  Thus,  A 
has  been  reduced  to  the  form  of  (13) . 


In  order  to  obtain  and  B,  similar  operations  must  be  done  on 

rows  of  the  reduced  matrix  A.  An  accounting  must  be  kept  on  the  column 
operations,  for  the  row  operations  must  be  done  in  the  same  order,  that 
is,  if  A has  been  postmultiplied  by  U^U_  . . . U , then  A must  be  pre- 
-1  -1  -1  ^ 

multiplied  by  . . . U2  . Note  that  the  row  operations  only  in- 
volve the  first  r columns,  for  the  remaining  n-r  columns  are  assumed  to 
be  zero  vectors.  Therefore,  if  a column  operation  involved  interchanging 
column  a with  column  m,  then  the  row  operation  would  interchange  row  £ 
with  row  m.  Similarly,  if  a column  operation • adds  -3  times  column  £ to 
column  m,  then  the  row  operation  would  add  3 times  row  m to  row  £. 


In  order  to  construct  3--j  the  vectors, 

ij 

from  which  3. . can  be  obtained  recursively, 
ij 


V^,  must  be  generated. 
From  (10) 


''i ' - S “ik''k  • 

k=l 


(A.,V^) 

“ik  = (V~,\y  • 


^k  ■ \ ^ 2,  . . . , 1-1, 

where  = A^.  Then  from  (19)  and  (20) 


(19) 


(20) 
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Therefore, 


Vi  • Ai  - c.iAj  - E - E 

i-1  i-1  k-1 

= - S^“ik\ 

i-1  i-2  i-1 

= A.  - E“i/.  + ECE  “ik^ki^N 

^ j=l  ^ j=l  k=j+l  ^ 

i-2  i-1 

=»  A.  - a.  . -A.  T - z2  Cot.  . - yZ  Qt-i  3i  OA.  . 
1 1,1-1  1-1  13  k=j*l  '‘3  1 


i-2 

la.  . - zL  a.,3i-  » j = l» 
11  ktj+i  Ik  kj  ' ■< 


C21) 


^ii  = 


'“ij 


, j = i-1 


.,  i-2 


C22) 


V.  SOME  ILLUSTRATIVE  EXAMPLES 


A.  Example  1, 

Let 


Then 


12  0 0 
1110 
2 3 10 

0 0 0 1 


'^l  = 


which  is  not  the  zero  vector. 


11 


where 


V = A - a V 
2 2 21  1 


“2 

'1 

" .5' 

1 

3 

1 

-.5 

3 

2 

2 

0 

0_ 

0 

0 

and  621  = <>21 


(Aj.Vj) 


2 + 1 + 6 + 0 


21  1 + 1 + 4 + 0 2 


''3 ' *3 


a V - a V = 
31  1 32''2 


'o' 

"1 

.5 

0 

1 

]_ 

1 

- (-1) 

-.5 

0 

9 

= 

1 

2 

0 

0 

0 

_0_ 

0_ 

0_ 

where 


31 


(A^,vp 

cw 


0 + 1 + 2 + 0 


32 


(A3,V2) 

(V2,V2) 


_ 0 + 0 + (-.5)  + 0 

.25  + .25  + 0 + 0 


-1 


and 


^31  “ “31  ” “32^21  ^ 


^32  “32 


Now  V3  is  the  zero  vector.  Therefore,  replace  column  3 with  column  4 
in  A.  We  will  assume  column  4 is  the  zero  column.  For  in  theory, 
times  column  1 and  -^^2  column  2 are  added  to  column  4.  Hence, 

the  reduced  A matrix  is 

2 0 o' 

1 0 0 

3 0 0 , , 

0 10 
12 


If  the  same  column  operations  are  applied  to  U,  which  was  initally  set 
to  the  identity  matrix,  then 


10  0-2 
0 10  1 

0 0 0 1 
0 0 10 


Since  was  the  zero  vector,  is  renamed  (interchange  columns) . A 
new  is  generated,  namely. 


'o' 

'o' 

0 

- o*v. 

- 0-V^  = 

0 

0 

1 

2 

0 

1 

1 

This  "new"  is  not  the  zero  vector.  Hence,  column  operations  terminate. 
The  row  operations  are  the  following: 


1 . Interchange  row  3 and  row  4 . 

2.  3^]^  times  row  4 added  to  row  1. 

3.  3^2  4 added  to  row  2. 

Thus , 


U'^AU 


5 8 0 0 

-1  -2  0 0 

0 0 10 

2 3 0 0 


The  rank  of  A is  3.  The  eigenvector  associated  with  the  zero  eigenvalue 
of  A is  the  fourth  column  of  U.  Since 


Q 


r 


“5  8 0" 

-1  -2  0 
0 0 1 


is  nonsingular,  no  further  reduction  need  be  applied, 
of  are  = 1,  = ^(3  + /IT)  and  X^  = y(3  - /IT)  . 

eigenvectors  are  the  following: 


The  eigenvalues 
The  respective 


13 


"o" 

1 

-y-ZT?" 

1 

-7+/IT‘ 

0 

’ ^2  “ 2 

2 

' ^3  = 2 

2 

_1. 

_ 0 _ 

_ 0 _ 

= 


To  obtain  the  eigenvectors  for  the  nonzero  eigenvalues  of  A,  (6)  is 
applied,  where 


B = [2  3 0]  . 


Thus, 


’0 

-4 

-4 

0 

X = i 

-1-/17 

X = i 

-i+ZIy 

0 

> -^2  4 

-5-/77 

’ 3 4 

-5+/r7 

1_ 

0 

_ 0 . 

^1  - 


and  the  eigenvector  for  the  zero  eigenvalue  is 


^4  = 


-2 

1 

1 

0 


the  last  column  of  U. 
B.  Example  2. 

Let 


3 -2  -1 
10  0 


0 10  0 
0 0 10 

It  is  easy  to  see  that  and  A^  are  independent  vectors  (also  V^, 

V2  and  V^)  and  A^  = A^^.  Thus,  the  only  column  operation  is  to  add  -1 
times  column  1 to  column  4.  Therefore, 

"3  -2  o' 

0^  = 1 0 0 
0 10 


and 
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B = [0 


0 


1]. 


where 


10  0-1 
y ^ 0 1 0 0 

0 0 10 
0 0 0 1_ 

If  we  apply  the  algorithm  again  to  Q^,  without  intermediate  calculations, 
we  have 


B'  = [0  1] 


and  U'  is  the  3x3  identity  matrix,  where  the  prime  denotes  the 
partitioning  of  Q^.  The  two  eigenvalues  of  are  = 2 and 

and  the  associated  eigenvectors  are 


Z 


1 


and 


The  respective  eigenvectors  of  are 


Z 


1 


r z’  1 

"4' 

1 

1 

2 

2 

_1_ 

and 


Z 


2 


r zi  1 

"i" 

U' 

2 

1 

x"^B'z; 

_i_ 

Finally,  the  eigenvectors  of  the  nonzero  eigenvalues  of  A are 
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7 
4 
2 
1 

and 

o' 

1 
1 

1_ 

The  eigenvector  associated  with  the  zero  eigenvalue  is 

-l" 

0 
0 
1 

that  is,  the  fourth  column  of  U.  Since  the  rank  of  A is  three,  this  is 
the  only  eigenvector  associated  with  the  zero  eigenvalue. 


VI.  CONCLUSION 


In  order  to  apply  this  algorithm  to  a computer  code,  an  a priori 
decision  must  be  made  for  a ” zero  vector’*,  due  to  machine  round-off. 

In  the  author’s  program,  which  generates  the  matrices  U,  and  B,  a 

vector,  V,  is  the  zero  vector  if 

CV,V)  < £, 

where  e is  some  preassigned  value. 

An  alternate  application  of  the  procedure  could  eliminate  some 
computer  code  ’’bookkeeping".  Since  matrix  multiplication  is  associative, 
a column  operation,  U^,  could  be  immediately  followed  by  the  row 

operation  This  would,  of  course,  increase  the  arithmetical 

operations,  since  the  knowledge  of  the  number  of  zero  columns  could  not 
be  totally  incorporated. 
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